1. Technical Field
This invention relates to the field of image and video processing, and, more particularly, to methods and systems for noise estimation and noise reduction of image sequences for video reception.
2. Discussion of Related Art
Image and video noise reduction has matured over the last two decades and has gained high recognition from the general public. Before the 1990's, the most common types of noise seen in images and video were film grain noise, recording tape noise, and analog white noise caused by signal transmission and analog electronic components. Still to this day, some motion pictures still show film grain noise although the quality of these films has improved over the years. When the digital transmission era arrived in the 90's, it basically replaced some of these older types of noise with newer types of noise such as image and video compression artifacts. Digital cameras using CCD technology produce a less grainy image than films but still have an intrinsic Gaussian white noise produced by its photo-electronic components.
The older types of noise are generally referred to as random noise and may also be referred to as 3-D dynamic noise in the video field. They are not related to image and video compression artifacts. However, image compression and video compression are known to change characteristics of the random noise from the original image or video data. In fact, some codes are very efficient at reducing random noise levels. This implies that automatic noise reduction systems using the latest noise estimation techniques can work effectively on different types of media, which may have undergone different types of compression.
Many systems and methods have been developed to automate random noise reduction processes. Some systems and methods are robust and may support a wide range of noise levels without having to adjust any level parameter.
U.S. Pat. No. 5,025,312 refers to transmission path noise. Its method involves extracting low-level picture detail, reticulating a fractional component of those details and recombining both versions to obtain an approximate noise signal. This approximated noise signal can then be subtracted from the image signal in non-moving areas. Selective coring is applied on moving areas to reduce temporal filtering artifacts. The method is locally adaptive and the processing level depends on the frequency response of a band-pass filter used in the low-level detail extraction.
U.S. Pat. No. 5,294,979 describes a system for estimating noise in a small number of horizontal scanning lines of a video signal which includes a low-pass comb filter applied on the luminance component. The system performs a weighted summation of the corresponding pixels of adjoining horizontal scan lines within the same field and transmits a plurality of weighted summations to an accumulator. The resulting noise estimates are local on the vertical axis of the image. Proper thresholding is required for differentiating noise from low-level textures so that only the noise signal is sent to the accumulator.
U.S. Pat. No. 5,657,401 describes a method and apparatus for measuring noise in an image sequence. The method divides the image into a plurality of blocks and compares the current input frame to the past frame. The resulting sums of absolute differences (SAD) compose a histogram. A bin with SAD interval [A, B] that has a high-enough count and has the lowest upper interval boundary is chosen to deduct an image-based global noise estimate. Since the method is based on the distribution in the generated histogram, it is possible to support a high range of noise reduction levels. However, the method proposed may not adapt well to varying noise levels intra image.
U.S. Pat. No. 7,317,842 describes a similar approach to estimating a frame-based global noise level. The image is divided into overlapping or non-overlapping blocks. For each block, the mean and standard deviation are calculated. The minimum standard deviation and corresponding block mean are found to set an interval. The minimum standard deviation may be adjusted based on the corresponding mean due to potential saturation (in very dark or very bright areas where the noise may have been clipped). The means of all standard deviations corresponding to the predetermined intervals (perhaps in the previous frame) are applied as the estimated noise level.
U.S. Pat. No. 7,515,638 describes a process for estimating both local and global noise in a sequence of images. The process involves techniques for motion estimation and compensation. The process uses displaced frame differences (DFD) to generate global statistics (such as minimum and mean) in order to estimate a global noise level. One adaptive algorithm can determine a local block-based noise level based on local DFD's of the current and previous frames, the global mean DFD, and the global noise estimate. The resulting local noise variance is then sent to a motion-compensated noise reducer. Although the process is fully adaptive and robust, it requires the costly addition of a motion estimator and motion compensator in an otherwise simple process.
U.S. Pat. No. 7,548,277 describes a similar device and method, but is limited to only performing a global noise estimate. The method also includes motion estimation and compensation techniques. It works on a pixel-based displaced frame difference, although the motion estimation and compensation methods are still block-based. Each pixel of displaced-frame difference is thresholded, validated, accumulated and accounted for in calculating the resulting global noise standard deviation. Luminance-based local weighting is then applied to this global estimate. Unless the device includes a motion-compensated noise reducer, adding a motion estimator and motion compensator may be too costly for the purpose of global noise estimation
However, none of these methods provide a sufficient locally-adapted global estimation of random noise levels, especially based on the image's local luminance intensity. Therefore, there is a need for an inexpensive and robust random noise estimation system for image and video.